CLASS PRESERVING AUTOMORPHISMS OF UNITRIANGULAR 

GROUPS 



VALERIY BARDAKOV, ANDREI VESNIN, AND MANOJ K. YADAV 

Abstract. LetUT„(_ft') beaunitriangulargroupoverafield A'andF^^fc := VTn{K)/ 
7fc (UT„ (/S')), where 74. {UT„(A')) denotes the fc-th term of the lower central series of 
VTniK), 2 < k < n. We prove that the group of all class preserving automorphisms 
of r„ k is equal to Inn(r„_fe) if and only if X is a prime field. Let gJi™' := UT„(Fpm ) / 
73 (UTn(Fpm)). We calculate the group of all class preserving automorphisms and 
class preserving outer automorphisms of gI*"^ . 



Let G be an arbitrary group. An automorphism a of G is called class preserving if 
a{x) e x'^ for all x d G, where x'~^ denotes the conjugacy class of x in G. The set of 
all class preserving automorphisms of G, which we denote here by Autc(G'), is a normal 
subgroup of Aut(G), the group of all automorphisms of G. Notice that Lin(G), the group 
of all inner automorphisms of G, is a normal subgroup of Autc(G). An automorphism /3 
of G is called normal if I3{N) C N for all normal subgroups N of G. The set of all normal 
automorphisms of G, which we denote here by Aut„(G), is a normal subgroup of Aut(G). 
Since every normal subgroup of a group G can be written as a union of certain conjugacy 
classes in the group G, it follows that every class preserving automorphism of G is a 
normal automorphism of G. Thus we get the following sequence of normal subgroups of 



The group of normal automorphisms for a given group G is a well studied object. See 
[IlIillTlIinillllllllinillSlIiniliniinilSl- We'n come back to this later in the last section. 

On the other hand Autc(G), the group of all class preserving automorphisms of a given 
group G, is a very less studied object. It seems that W. Burnside is the first person who 
talked about class preserving automorphisms and posed the following question in 1911 [21 
p. 463]: Does there exist any finite group G such that G has a non-inner class preserving 
automorphism? In 1913, Burnside himself gave an affirmative answer to this question [3]. 
He constructed a group G of order p®, p an odd prime, isomorphic to the group 



where Fp2 is the field consisting of elements. He proved that Autc(G) is an elementary 
abelian p-group of order p^, but | Inn(G)| — p'^. 

In 1947 G.E. Wall [SJ constructed examples of finite groups G such that 
Inn(G) < Autc(G). In 1966, C.H. Sah ^23^ studied the factor group 



1. Introduction 



Aut(G) 



Inn(G) < Autc(G) < Aut„(G) < Aut(G). 
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Outc(G) :— Autc(G)/ Inn(G) for an arbitrary finite groups G, using cohomological tech- 
niques. After this, Autc(G) was studied by M. Hertweck [9] in 2001 for a special class of 
groups G and by the third author [3S] in 2007 for finite p-groups G. In 2005, G. Endimioni 
[B] proved that Outc(G) = 1 for all free nilpotent groups G. 

From this point onwards we'll mainly talk about unitriangular groups. Let UT„(_fC) 
denote the unitriangular group consisting of all n x n unitriangular matrices having entries 
in a field K, where n > 2. Let Tn.k denote the factor group UT„(i^)/7fc(UT„(i^r), where 
1 < k < n. Obviously r2,A:, r„^i and Tn,2 are abelian and therefore all class preserving 
automorphsms of these groups are inner. 

Since the first example of a group G such that Inn(G) < Autc(G) comes from unitriag- 
ular groups, it is quite natural to pose the following problems: 

Problem A. Find necessary and sufficient conditions on Tn,k, n > 3, 3 < k < n, such 
that Autc(r„^fc) = Inn(r„_fc). 

Problem B. Let G = UT„(i^) be such that Inn(G) < Autc(G). Study the stucture of 
Aute(G). 

In this paper we study these problems. We provide a solution to Problem A in the 
following theorem. 

Theorem A. Let r„,fe = \JT„{K)/jkiVT„{K)), where X is a field, n > 3 and 3 < fc < n. 
Then Autc(r„jt) — Inn(r„^fe) if and only if ii' is a prime field. 

The statement follows from Theorems [T] and [31 which we prove in Section 2. Alterna- 
tive proof of Theorem [T] based on the ideas suggested by N. S. Romanovskii is given in 
Appendix A. 

On the lines of the proof of Theorem^ it follows that Autc(UT„(Z)) = Inn(UT„(Z)) 
for every positive integer n, where Z denotes the ring of integers. 

Next we study Problem B for the following special case. For integers n > 3, to > 1 and 
a prime p, let UT„ (Fpm ) denote the group of all n x n unitriangular matrices with entries 
in the field F^™. Set G^"^ := UT„(Fp™)/73(UT„(Fp™)), where 73 (UT„(Fp™)) denotes 
the third term of the lower central series of UT„(Fpm). In the following theorem, which 
we prove in Section 3 as Theorem 3.2, we calculate the group Autc(Gi™'') and notice 
(surprisingly) that | Autc(Gl'"^)/Inn(G^'"^)| is independent of n. 

Theorem B. Let Gn""* be the group defined in the preceding paragraph. Then Autc(Gl™'' ) 
is elementary abelian p-group of order p2m^+mn-3m^ Moreover the order of Outc(G^™'') 
is p2m(m-i)^ which is independent of n. 

We would like to remark that the group of all automorphisms of UT„ (K) was studied 
by Levchuk [T3] . 

In Section 4 we give a quick survey on normal automorphisms of groups (defined above) 
and pose several interesting problems in the sequel. 

Before concluding this section we give some definitions, which we'll use many times 
in the paper. Let G be a group with a fixed set of generators {xi, ■ ■ ■ ,Xd} (say). An 
automorphism ip is called basis conjugating if (p{xi) £ xf, for all i such that 1 < i < d. Let 
Cb(G) denote the group of all basis conjugating automorphisms of G. An automorphism 
of a group G is said to be central if it induces identity on G/ Z(G), where Z(G) denotes 
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the center of G. We write the subgroups in the lower central series of G as 7„(G), where 
n runs over all strictly positive integers. They are defined inductively by 

71 (G) = G and 7n+i(G) - [in{G), G] 

for any integer n > 1. 

2. Unitriangular groups over fields 

In this section we consider the group UT„(iir) of lower- unitriangular matrices over a 
field or ring K . It is well-known that this group is generated by elementary transvections 
tikW, where 1 < k < i < n and X G K \ {0}, which satisfy the following relations (see 
m § 3]): 

{tij {a(3), ii k — m, 

tjnk{-a(3), if i = j, 
In, if fc ^ TO and i ^ j, 

where 2 < i,m < n and 1 < fc,j < n — 1. Here [a,b] — a^^b^^ab and 1„ is the unity 
matrix of degree n. 

Notice that the group \JTn{K) is generated by ti+i,i{K) for 1 < i < n — 1, where 
ti+i^i{K) is the subgroup of UT„(_ft') generated by {ti+i^i{a) \ a e K}. Thus 

UT„(i^) = {hAK),t3,2iK),---,ta^n-l{K))- 

Starting with the case ii' = Q, we observe that the additive group (Q; +) is infinitely 
generated. It is generated by the elements of the form |, where p runs over all prime 
integers. So, the subgroup ti+i^i{Q) of UT„(Q), where 1 < i < n—1, is infinitely generated 
by the elements where p runs over all prime integers. But, in the following 

lemmas, we show that if is a basis conjugating automorphism of UT„(Q)/7fe(UT„(Q)), 
then the image of ti+i_i(Q) (modulo 7a;(UT„(Q))) under (p is determined by the image of 
the element tij^i_i(a/b) for some arbitrarily chosen fixed rational number a/6. 

Here and below we use the same notations for transvections in VTniK) and for their 
images in UT„(i^)/7fe(UT„(i^)), where 2 < k < n. 

Lemma 1. Let ip be a basis conjugating automorphism o/ UT„(Q)/73(UT„(Q)). Then 
for any i the action of tp on the subgroup ti+i^i{Q) (modulo 73(UT„(Q))j is uniquely 
determined by the image of ti+i^i{^) for an arbitrarily chosen non-zero rational number 
a/b e Q. 

Proof. Let us fix a non-zero rational number a/b G Q and suppose that 
^{ti+iA{a/b)) = ti+i,i{a /b) ti+i^i-i{X) ti+2,i{l^) 

for some A,/i G Q, where 1 < i < — 1, and we set <i_|_i_i_i(A) — 1 when 1 = 1 and 
ti+2.i{P') = 1 when i = n — I. To complete the proof of the lemma, we need to find 
f(ti+i,i{r / s)) for any rational number r/s. This simply means that we need to find 
x,y (depending only on a, b, A, fi, r and s) such that 

(piti+i^i{r/s)) = ti+i,i{r/s) ti+i^i^i{x)ti+2,iiy)- 

The obvious relation (ij+i,i(r/s))-'" = (<i+i,i(a/6))'"', gives us 

^i{t,+,4r/s))n ^fiiu+i Aa/b)r), 
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which is equivalent to 

Thus comparing the corresponding entries of the matrices on both sides, we get 

asx = brX, asy = brfj,. 

This gives the required values 

brX brfi 

X = , y = , 

OS as 

which completes the proof. □ 

In the following lemma we consider basis conjugating automorphisms if of the group 
UT„(Q)/7fe+i(UT„((Q)) such that for each i, (p{ti+i^.,{wi)) = , where gi = 

tk,iixi,i) tk+i^2ixi,2) ■ ■ ■tn,n-k+iixi,n-k+i)- The proof of this lemma goes on the lines of 
the proof of Lemma [TJ 

Lemma 2. Let (p be an automorphism o/UT„(Q)/7/j+i(UT„((Q)) such that for k < n — k, 

{ti+i,i{wi) ti+k,i(-Xi.i+iWi), if i < k; 

tt+i,i{wi) ti+i^i-k+i{xi^i-k+iWi) ti+k,i{-Xi,i+iWi), if k < i < n - k; 
U+i,ii'^i) U+i,t-k+i{xi^i-k+iWi), ifi>n~k, 

and for k > n — k, 

{ti+i,i{wi) ti+k.i{-Xi.i+iWi), ifi<n-k; 
ti+i^i{wi), ifn-k<i<k; 
ti+i,i{^i)U+i,i-k+iixi.i-k+iWi), if i>k, 

where Wi,Xi,i+i, Xi^i-k+i G Q- Then for any i, the action of on the subgroup ti+i.i(Q) 
(modulo 7/c+i (UT„(Q)) ) is uniquely determined by the image o/ti-|_i.i(|-) for an arbitrarily 
chosen non-zero rational number a/b G Q. 

In the following theorem, we prove the sufficient part of Theorem A. 

Theorem 1. Let Tn.k — UT„(if)/7fc (UT„(if)), where K is a prime field, n > 3, 2 < 
k<n. T/ieK Autc(r„^fc) Inn(r„^fc). 

Proof. Let us fix a positive integer n > 3. Since X is a prime field, then K ^ ¥p ov 
K = Q. The nilpotency class of Tn^k is {k — 1) and Z(r„^fc) = is generated by 

the subgroups 

tk,l{K),tk+1^2{K), . . . , tn^n-k+l{K). 

It is easy to see that Tn,k/ Z(r„,fe) = r„^fc_i. 

We use induction on k to prove Autc(r„^fe) — Inn(r„^fc), where 2 < k < n. li k ~ 2, 
then r„.2 — K"^^^ is abelian and the statement holds trivially. Before going to the 
inductive step, we compute the case when fc = 3 just to show how things work. Let 
if G Autc(r„^3). Then it follows from Lemma [T] that (p is completely determined by its 
images on ii_|_i^i(s), where s = 1 ii K = ¥p and s = l/m, for some fixed prime m, if 
K = Q. Let g = t2,i("i) ^3, 2(012) ■ • • in,n-i(ari-i), where ai,a2, - ■ ■ ,an-i G K. Since the 
nilpotency class of r„.3 is 2, for a fixed s, we have 

(^2,1(5))'' = i2,i(s)i3,i(-a2s), 
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(^3,2(3))^ = t3,2(s)t3,i(ais)f4,2(-a3s), 

= *i+i,i(s)ti+i,i-i(ai-is)ii+2,i(-Q!j+is), 

{tn-l,n-2{s)y = tn-l,n-2{s)tn-l,n-3{an-3S)tn,n-2{-an-ls), 
{tn,n-l{s)y = tn,n-l{s)tn,n-2{an-2S). 

Since (p G Autc(r„^3) and the nilpotency class of is 2, there exist X2,X3,..., Xn-i, 
2/1, 2/2, • • • , 2/n-2 e K such that 

¥'(i2,l(s)) = ^2,l(s)*3,l(-2/l'S), 

¥'(i3,2(s)) = t3,2(s)i3,i(a:;2s)t4,2(-2/2s), 

lyj(ii+l,i(s)) = ti+i^i{s)ti+i^i-i{XiS)ti+2,i{-yiS), 

^{tn-l,n-2{s)) = tn-l,n-2{s)tn-l,n-3{Xn-2s)tn,n-2{-yn-2s), 
V{tn,n-l{s)) = tn,n-l{s)tn,n-2{Xn-ls). 

Automorphism ip preserves all conjugacy classes in r„^3, so, in particular, it also pre- 
serve the classes of elements hi = ti+i^i{s) tt+3,j+2,(s), where i = 1, . . . , n — 3. Acting by 
<p we have 

ip{hi) = <fi{t2,i{s)t4,3{s)) = i2,i(s)t4,3(s)i3,i(-2/is)i4,2(a;3s)i5,3(-2/3s)- 

Since hi and <p(/ii) are in the same conjugacy class, <p(/ii) = /if^ for some element 
gi G r„,3, say gi = t2,i{al) t3,2(a!2) • ■ • tn,n-i{(xi-i)- Now 

/if = {t2,l{s)t4As)r 

= t2,i{s) ^4,3(5) t3,i{-als) t4,2iQ.ls) h^3{-als). 
From (fi{hi) = /if\ by comparing the corresponding entries of the matrices we get 

(1) yi = X3- 

Moreover, = yi, a\ = 2/3, and for j ^ {2,4}, numbers aj can be taken arbitrarily. 
For 2<z^n — 4we have 

(p{hi) = (/3(t,:+i,i(s)ii+3,j+2(s)) 

= U+l,ii^) ti+l,i-liXiS) ti+2,i{-yis) U+3,i+2{s) tj+3,i+l(^J+2s) U+A.i+2{-yi+23) 
= U+lAs) U+3,i+2{s) ti+l,i-l{XiS) ii+2,i(-J/iS) U+3,i+l{Xi+2s) ti+A.i+2{-yi+2s) ■ 

Since hi and ip{hi) arc in the same conjugacy class, <p{hi) = /i^' for some element gi G r„^3, 
say gi = t2s{(A)t3,2{a\) ■ • • t„,„-i(a^_i). Now 

/if = {ti+l4s)U+3,^+2{s)y' 

= ti+i^s) ti+i,i-i{al_-t^s) ti+2,i(-Q!-+iS) ii+3,i+2(s) ii+3,i+l(a^+iS) ti+4,i+2(-a^+3S) 

= ti+i,i{^) ti+3,i+2{s) (a-_is) ii+2,i(-Q!-+is) ij+3,i+i(a-+is) ti+4,^+2{-a'■_^_3s). 

Prom <p(/ij) = /if, we get 

(2) yi = Xi+2 
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for i = 2, . . . , n — 4. Moreover, al_^^ = yi, = yi+2, and for j ^ {i — l,i + l,i + 3}, 
numbers can be chosen arbitrarily. 
Finally, we have 

— tn-2,n-3{s) tn,n-l{s) tn~^2,7i-'i{Xn-3s) tn-l,n-3{~yn-3s) ^n,n-2 (^^ri- 1 s) • 

We have <p(/i„-3) = hl"L^ for some element g„_3 e r„,3, say gns = hActi"^) i3,2(a2^^) 
•••tn,„-iK:?). So 

C-t = {tn-2,n-3is)t„,n-l{s)y"-' 

= in-2,«-3(s) in-2,n-4(a^l4s) t„_i_„_3(-a"l^s) in,„_l(s) i„,„-2 (Q!^I2s) 
= in-2,n-3(s) i„,„_l(s) <n-2,n-4(a"l4s) in-l,n-3(-a"l2'S) in,n-2 (a"l2S) ■ 

Therefore, by comparing the corresponding entries of ip{hn~^) and (/i„„3)3""^, we get 
(3) y„_3=x„_i. 

Moreover, a^I^ = x„_3, hence by © aj^l^ = ?/„_5, and q;"I2 = y„-3- For j ^ {n - 
4, n — 2} numbers can be taken arbitrarily. 

Thus, it follows from ([J), ^ and dS]) that by defining 

ai = X2, a2=yi, a3 = 2/2, Q^n-i = yri-2 

and g = i2,i(ai) i3,2(a2) • • • in,ri-i(a«-i), we obtain 

(p{ti+l^^{s)) = (ii+i^j(s))^, i = 1, . . . , n - 1. 

This proves that ip is the inner automorphism determined by g. Hence Autc(r„^3) = 
Inn(r„_3) 

Let us now assume that Autc(r„^fc) = Inn(r„^fc) for some fc > 3. The natural homo- 
morphism r„^fc_|_i — > r„.fc induces the homomorphism 

Aut(r„,fc+i) — > Aut(r„,fc). 

Let Tp denote the image ip £ Aut(r„^fc+i) under this homomorphism. 

Suppose that ip S Autc(rn^fe+i). Then Ip G Autc(r„,fe). By the induction hypothesis 
there exists an inner automorphism aJ G Inn(r„^fc) such that Tpuj is the identity automor- 
phism of r„^fc. Let uj £ Inn(r„^fc+i) be the pre-image of uJ. Then ip := ip uj is a central 
automorphism of r„.fc+i. Since ip is central as well as class preserving, it acts on every 
<i+i_i(a), a e if by conjugating it by some element Xi of the form 

tk,l{Xi^l) tk+l,2{Xi,2) ■■■ tn,n-k+l{Xi^n-k+l), 

where i = l,2,...,n — 1 and Xij e K. By Lemma [H it is sufficient to study ip on the 
generating elements ti+i^i{s), where s = 1 ii K = ¥p and s = 1/m, for some fixed prime 
m, a K ^ Q. Notice that tp is defined on these generating elements in the following way: 
For k < n — k, 

{U+ia{s) ti+fc,i(-a;i,i+is), if i < fc; 

ti+is{s) U+iA-k+i{xi,i-k+is) ti+k,i{~Xi^i+is), ii k <i <n - k; 
U+id{s) ti+i^i-k+iixt^i-k+is), if i > n - fc. 
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and for fc > n — fc, 

ti+i,i{s) ii+fe,i(-a;i,i+is), if i < n - k; 
= s i{n-k<i<k; 

Also notice that an inner automorphism fg determined by 

9 = tfe,i(Q!i)tfc+i,2(Q!2) • • ■ tn,n-k+i{oin-k+i), oij & K for I <3 <n-k + l, 

is defined on the generating elements in the following way: 
For k < n — k, 



fg{ti+l,i{s)) 



ti+fc,i(-Q!i+is), if i < fc; 

*i+i,i(s)ti+i,j-/s+i(Q!j-/s+is)ti+/s,i(-Q!i+is),if k <i <n - k; 
*i+i,i(s)ii+i,i-fc+i(a!i-fc+is), i£i>n-k, 



and ioi k > n — k, 

fg{ti+l,i{s)) = < 



ti+i.i{s) ii+fe,i(-Q!i+is), Hi <n - k; 
ti+i^i{s), \i n — k < i < k] 

ti+i,i{s) ti+i,j_fe+i(ai_fe+is), if z > k. 



Let us consider the elements 

hi^k — ti+is{s)ti+k+l,i+k{s), 

where i = 1,2, . . . ,n — k — 1. Then for A: < n — A; 

ti+l,i{s) ti+k+l,i+k{s) ti+k,i{ — Xi^i+ls) ti+k+l,i+l {Xi+k,i+is) 

■ti+2k,i+k{-Xi+k,i+k+is), if i < fc and i + k < n - k; 

ti+l,i{s) ti+k+l,i+k{s) ti+k,i{ — Xi^i+ls) ti+k+l,i+l ixi+k,i+is), 

if i < k and i + k > n — k; 

V'(/ii,fe(s)) = { ti+ij{s)ti+k+l,i+kis)ti+i^i-k+l{Xi^i-k+ls)ti+k,ii — Xi,i+ls) 
'ti+k+l,i+l{Xi+k,i+ls) ti+2k,i+k{^Xi^k,i+k+ls) , 

ii k < i < n — k and k<i + k<n — k; 

ti+l,i{s) ti+k+l,i+k{s) ti+i^i-k+1 (Xi^i-k+is) ti+k,i{-'Xi,i+ls) 

■ti+k+i,i+iixi+k,i+is), iik<i<n-k and i + k> n-k, 
and for fc > n — fc, 

i'{hi,kis)) = ii+l,i(s) ti+k+l,i+k{s) ti+k,i{-Xi,i+ls) ti+k+l,i+l{Xi+k,i+ls), 

ioT all i < n — k — 1. 

The reason why we are considering hi^k = U+i,i{s) ti+k+i,i+k{s) is that and 
'ip{tj+ij) both involve conjugation by elements from the same subgroup {ti+k,i+i{K)) 
only when \i — j\ = k. More precisely, when i + k = j , i^{ti+i_i) and i^{tj+ij) involve 
and respectively. If \i - j\ ^ k, then ti+i,i(s) and 

ti+i^i{s) centralize the subgroups tjj^k,j+i{K) and tiJ^k,i+i{K) respectively. 

So to prove tf) is inner, it is sufficient to prove that ajj^j+i = Xi-^.k,i+i iov 1 < i < n—k—1. 
Since V is class preserving, there exists an element 

Qi = tk,l{yi,l) tk+l,2{yi,2) ■ ■ ■ tn,n-k+l{yi,n-k+l) 

such that tp{hi^k) = hl\ for 1 < i < n — fc — 1. Now we calculate /i?*^.. 
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For k < n — k 



ti+i.i{s) <i+/£+i,i+/c(s) ti+k,i{—yi,i+is) ti+k+i,i+i{yi,i+is) 



■ti+2k,i+k{-yi+k,i+k+is), Hi < k and i + k < n - k; 

ti+i,i{s) ti+k+i,i+kis) ti+k,i{—yi.i+is) ti+k+i.i+i{yi,i+is), 



ii i < k and i + k > n — k; 




ti+k+l,i+k{s) ii+l,i-fe+l(2/i,i-fc+is) ti+k,i{ — yi,i+is) 
■ti+k+l,i+l {yi,i+lS) ti+2k,i+ki — yi+k,i+k+is), 



ii k < i < n — k and k<i + k<n — k; 



ti+l,i{s) ti+k+l,i+k{s) ti+l,i-k+liyi,i-k+is) ti+kA{ — yi,i+ls) 



, ■ti+k+i,i+i {yid+is), iik <i <n - k and i + k > n - k, 



and for k > n 



k 



for alH < n — fc — 1. 

Since ^p{hi^k) — h^\, by comparing the corresponding entries of the matrices we get 



Hence x^.i+i = a^i+fe.i+i , which proves that ip is an inner automorphism of r„_fc_|_i , induced 
by 5 = ^fc+i,2(a2) • • • iri,n-fe+i(an-fc+i), where ai = Xfc,i, a2 = a;i^2, ai = 

Xi-i^i, an-k+i — Xn-k.n-k+i- We would hkc to remark here that conjugation 

by tk,i{,Xk,i) appears only in V'(ifc+i,fc(s)) and conjugation by tn^n-k+i{xn-k,n-k+i) ap- 
pears only in ij}{tn^k~i,n~k{s)). That is why we have taken ai — Xk,i and an-k+i = 
Xn~k,n-k+i- Sincc ip = Lf uj with uj e Inn(r„^fc+i), it follows that Lp is an inner auto- 
morphism of Tn^k+i- As Lp was an arbitrary class preserving automorphism of r„jt+i, we 
have Autc(r„^fc_|_i) — Inn(r„jt+i) for all fc > 3. □ 

The proof of the following theorem goes on the lines of the proof of Theorem [TJ 

Theorem 2. Autc(UT„(Z)) = Inn(UT„(Z)), where 7L is the ring of integers and n is 
any positive integer. 

Now we prove the necessary part of Theorem A. 

Theorem 3. Let r„ ^ — 'UTn{K)/jk (UT„(iir)), where K is not a prime field, n > 3, 
3 < fc < n. Then Autc(r„jt) ^ Inn(r„ fc). 

Proof. It is sufficient to construct a non-inner class preserving automorphism of r„ fc. 
Consider K as a vector space over Kq, where Kq is its prime subfield, with the basis 
{e", e^, e^, . . .}, where e" — 1. Then the group ti,j{K) is generated by elements tij{e^), 
1 = 0,1,..., if Kq = ¥p and by elements tij{-e''), 1 = 0,1,... and q runs over all primes, 



By Lemma [H to define an automorphism tp of the group UT„ (K) of the form given 
below, it is enough to define its images (p{ti+i_i{e^)) and (p{ti+i^i{^e^)) for I = 1,2,.... 
Let us consider the following map ijj of r„^fc into itself: 



UK, 







i;{tk,k-i{-e')) 



tk,k-i{e°)tk,i{e°c), 
tk,k-i{-e^) for j > 1, 
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V'(tj+i,i(-e')) = ii+i^,,(-e') for i 7^ /c- 1 and Z = 1,2, 

where c is a non-zero elements of K and q runs over all primes. It is easy to see that '>p 
is a central automorphism of r„ fc. 

First we show that ■(/i is class preserving. Any element 5 G G can be represented in the 
form 

g = ^2,1(01) • • • tk,k-i{ak~i) ■ ■ ■ i„,„-i(a„_i) d, 

where d G 72(r„.fc). If afc-i = 0, then ip fixes g. So assume that ak~i 7^ 0. Since the 
automorphism ip is central, it fixes every element of j2{^n,k)- Thus by the definition of 
ip, we get 

for some (3 G K, since ifc,i(/3) G Z(r„ fe). Let h = ffc-i.i(/?/aA:_i). Then 

{tk,k-iiak-i))''' = ifc_i,i(-^/Q!fe_i) tfe,fc_i(afc_i) tfc_i,i(^/afe_i) 
= ^fe,fe-i(afe-i) 

and for any u d K, 

{UAu)f^trAu) if (r,s) ^ (fc,fc-l). 

Therefore, = which shows that i/j is class preserving. 

Now we show that -0 is not inner. Let us assume the contrary, and therefore there 
must exist some h G r„.fc such that 

(4) (ifc,fc-i(e°))" = hM-iie") tkAe'^c), 

(5) {tk,k-i{^eJ)f = tk,k^i{^e^)hTj>l, 

(6) (i,+i,,(ie'))'* = t,+i,,{-e^) for i^k-lsiud 1^0,1,..., 

where (7 runs over all primes. Taking into account ([4]) and (jG]), notice that the only 
possibility for the value of h is tk-i,i{c) modulo Z{Tn,k)- Now by inserting the value of h 
in ([5]), we get 

{tk,k-i{^e^))tk,i{^e^ c) = tk,k-i{^e^) for j > 1 and each prime q. 

Since ^ 0, it is possible only when c = 0, which contradicts our choice of c being 
non-zero. Hence is not inner. This completes the proof of the theorem. □ 

Corollary 1. Let r„,fc = UT„(Q[x])/7fc(UT„(Q[x])). Then Aute(r„,fc) Inn(r„,fc), 
where n > 3, 3 < k < n. 

Proof. We can consider Q[x] as an infinite dimensional vector space over Q with basis 
e" = 1, = X, — , . . .. Thus the proof follows on the lines of the proof of Theorem 

m □ 



We remark that the case n = 3 of Theorem A was proved by another techniques in [8] . 
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3. Quotient groups of unitriangular groups 
over finite fields 

Throughout this section K always denotes the field Fp™, where m is a positive integer. 
Let us denote by g[i^\ n ^ 3, m ^ 1, the quotient group: 

Gi™) =UT„(Fp..)/73(UT„(Fp^)). 

Obviously G^™^ is a nilpotent group of class 2. 

The group Gg^^ is known as the Heisenberg group modulo p. It is a group of order 
with two generators x, y (say) and relations 

z — x^^y^^xy, x^ — y^ — — 1, xz — zx, yz — zy. 

It is easy to check that \dn \ = p^n-s ^^^^ \Z{dn^)\ = p"-^ so 

lInn(G«)| = |GW/Z(GW)|^p"-i and |Cb(GW)| = /"-i)^"-^). 

(2) 

We remark that Gg is the group constructed by W. Burnside, which is mentioned in 
the introduction. 

Let us represent the elements of the field Fp™ in the following form: 

Fpn. = {ao ■ 1 + ai ■ 6* + 02 ■ + • ■ • + a„_i ■ 6*™-! | a, e Fp}, 

where is a root of the minimal irreducible polynomial over Fp. Then gI™'' is generated 
by the transvections 

ti+i^i{9'^), i = l,...,n-l, fc = 0, ...,TO-1 
of order p. The commutator oita^piO'^) and t^^s{9^) for a > /3, 7 > J is defined as follows: 

f t^AS''+'), if/3 = 7, 

[ 1„, otherwise. 
Here and below we use the same notations for transvections in UT„ (Fpn. ) and for their 

^(m) 

images m G„ . 

A group G is said to be Camina group if x'^ = X72(G) for all x G G \ 72(G). The 
following lemma can also be derived as a corollary of [321 Theorem 5.4] by noticing that 
Gg™'' is a Camina special p-group. But we here give a direct simple proof. 

Lemma 3. Any basis conjugating automorphism o/Gg™"*, to ^ 1, is class preserving, 
I.e., Cb(G^"^) = Aut,(G^™^). 

Proof. The group Gg™^ is generated by 2m elements 12,1(0'') and ^3,2(6''°), where k = 
0, . . . ,TO — 1. Notice that any automorphism ip G Cb(G3'"'') acts on the generators as 
follows: 

V{t2A{9')) = i2,l(^')^3,l(-y'')), 

for some cc''^-' , y^'^-' G Fp™, where fc = 0, . . . ,to — 1. It is easy to see that ip is a. central 
automorphism of Gg™'' . The automorphism (p is class preserving if and only if for each 
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h € Gg™^ there exists an element g such that ^p{h) = h^. Let h — t2,i(Ai) t3,2(A2) t3,i(/^i) 
be an arbitrary element of . Since ip is a central automorphism, we have 

^(i2,l(Al)) = t2,l{Xl)t3,l{-P), 
V{t3,2){X2) = t3,2(A2)t3,l(<3) 

for some P,Q € Vp^ . Thus 

^{h) = t2,l(Al) t3,2(A2) t3,i{-P + Q + Ml). 

Suppose that there exists g e such that ip{h) = h^. The element g can be represented 

in its normal form as 5 = ^2, 1(0^1) *3, 2(^2) i3,i(/3i) for some ai, a2, Pi G Fpm. Consider 
the conjugation of g on the generators of Gg^^ which is given by 

(t2,l(^'^))^ = t2,l(^?')t3,l(-a2^'=), 

Therefore 

= t2,l(Al)t3,2(A2)i3,l(Q!iA2 -Q!2Al + Ml)- 

Thus ip{h) = if and only if 

aiA2 " a2Ai = —P + Q. 

But for any given P, Q (depending on (p) and for any given Ai, A2 (depending on h), one 
can always find ai and a2 such that q;iA2 — a2Ai = —P + Q. Hence any basis conjugating 
automorphism of Gg™^ is class preserving, which completes the proof of the lemma. □ 

Now we prove Theorem B. 

Theorem 4 (Theorem B). Let G^™'' be the group defined above. Then Autc(Gl™'') is 
elementary abelian p-group of order p^"^ +mn-3m_ Moreover, the order of Outc{G^^) is 
p2m{m-i) ^ yjfij^(.fi j^g independent of n. 

Proof. As mentioned above, is generated by {ti^i^i{6^) |l<i<n — 1, 0<fc< 
m — 1}. Observe that 

(i/3+l,/3(y))*"+^'"'^^ = t^+l,;3(2/)[i/3+l,/3(2/),ia+l,a(a;)] 

t/3+i,/3(2/)i/3+i,/3-i(?ya:), if/3 = a + 1, 
</3+i,/3(j/)i/3+2,/3(-2/a;), if/3 = a-l, 
i/3+i,/3(y), otherwise. 

Let g S gI"*^ be an arbitrary element. Then it can be written in the normal form: 

g = i2,l(Q!l)*3,2(Q!2) • ■ ■tn,n-l{an-l)t3,l{Pl)ti,2{P2) ' ' ■tn,n-2{Pn-2)- 

Since gIT'^ is nilpotent of class 2, for any j = 1, ... ,n — 2, the element tj+2,j{l3j) lies 
in the center of G^\ So while considering the conjugation action by g, we can assume, 
without loss of any generality, that 

g = t2,l(ai) i3,2(Q!2) • • • tn,n-l{0!n-l)- 
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Thus g acts on the generators of Gn™'' as follows: 

ihAS''))' = i3,2(0')t3,l(ai^'=)k2(-a3e'=), 



{tn-l,n-2{0'^)y = tn-l,n-2iO'')tn-l,n-3ian-30'')tn,n-2{ — an-20''), 
{tn,n-l{0'')Y = tn,n-l{6'')tn,n-2{otn-20'^), 

where fc = 0, . . . , m — 1 and ai 6 Fpm for i = l,...,n— 1. 

Let be a basis conjugating automorphism Then can be defined on the 

generating elements of as follows: 

¥'(t2,i(^')) = i2,i(0'=)t3.i(-2/f^ 

¥'fe,2(^1) = i3,2(^'=)i3,l(4'V4.2(-J/f ), 



'p{tn-l,n-2{0^)) = tn-l,n-2iO'')tn-l,n-3ixi%)tn,n-2{-yi%), 



¥'(in,n-l(^'=)) = t„,„-l(e'=)i„,„-2(xi'=2i), 



for some elements xf'^ and of Fpm, where i = 2,...,n — 1, j = 1, . . . , n — 2, and 
= 0, . . . , m — 1. 
This shows that 

|Aute(GW)| < |Cb(GW)| <p™^(2n-4)^ 

since each Xi as well as yj can take at most values in Fp^ . We are now going to show 
that if is a class preserving automorphism of G[i^\ then many of and yj*^-* depend 
on one another. So assume that ip \s & class preserving automorphism of Gn^^\ Let us 
consider the elements 

where p, g G {0, 1, . . . , m — 1} and i = 1, . . . , n — 3. 
Then 

= i2,l(^^)t4,3(^'')i3,l(-2/|^V4,2(4'^)t5,3(-2/3''^ 
ii+2,i(-yl^^) ti+3,i+l(4+2) ii+4,i+2(-J/l+2) 

for 1 < i < n — 3, and 

^(K^Z3)=*n-2,n-3(0^)tn,n-l(^')t„-2,„-4(xi^23)*n-l,„-3(-yi"i3)^n,„-2(xi^ll). 
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Since is class preserving, for each ft,^'* there exits gf'"* e Gn such that (p{h^''^) = 
{hfyr. Let = t2,iKf)i3.2K'|) ••• t„,„-i(af;^i), where 1 < z < n - 3 and 
< p, q < m ~ I. 

The conjugation action of gf'"^ on h^'"^ is given by 



P,9 , 



iKy^^ = t.+l.(0^)t.+3.+2(e^)t.+M-l«tie^) 

for 1 < i < n — 3, and 



•i„_l,„_3(-a^'!3,„-2 in,n-2(a^'!3,„-2 ^'')- 



Since ip(hF^'^) — (/if'')^^ by comparing the corresponding entries of the matrices we 
get 

(7) y^)6'^^x^},e~\ 

where p,q € {0, 1, . . . , m — 1} and i = 1, . . . , n — 3. We remark that this set of equations 
doesn't contain parameters x^2^^ and yf/!^2 ^'^^ k ~ 0, . . . ,m — 1. Putting q — in ^ we 
get 

(8) ^/l^^--^?2^^ 

where p = 0, . . . , m — 1 and i = 1, . . . , n — 3. Now putting p = in (l7|) we get 
which, along with ([8]) for p — 0, gives the following set of equations 

where g = 1, . . . , rn — 1 and z = 1, . . . , n — 3. Thus it follows from ([8]), (jll) and the above 
remark, that depends on the following 2m + n — 3 parameters only: 

a;|_j.'*2, i = 1, . . . , n — 3, ' , yi^2' fc = 0, . . . , m — 1. 

Hence 

I Autc(Gi'"))| ^ p2m^+™"-3m^ 

since each of these parameters can have at most values in Fpn. . 

Now we proceed to show that p2m^+mTi-3m <^ | Autc(Gl'"^)|. Fix fc and i belonging to 
{0, 1, . . . , TO — 1}. Suppose 

and (j)^'^{ti+i^i{9^)) = ti+i^i{9^) if ^ (2,fc). It is easy to see that (p'''^ is a basis 

conjugating automorphism. Notice that 0*^'^ induces a basis conjugating automorphism 
on the subgroup consisting of 3 x 3 top left blocks of all matrices of G'i™'' and it fixes 
every element outside this subgroup. Hence, by Lemma [3l (f>'^'^ is class preserving. Let 
Ai denote the subgroup of Aut(GL''*^) generated by {^'^'^ | < fc,^ < to — 1}. Thus we 
get class preserving automorphisms of G^™^ . 
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Again fix k and £ belonging to {0, 1, . . . , to — 1}. Suppose 

^'''(i„-i,„-2(e')) = (^„-l,„-2(^^'))*"^"-l(^') =i„-i,„-2(e")t„,„-2(-^''+') 

a.nd'ip'''^{ti+is{9^)) = ti+i,i(6'-') if (i, j) ^ {n—2,k). It is again easy to see that V''^'^ is basis 
conjugating automorphism. Notice that ip'''^ induces a basis conjugating automorphism 
on the subgroup consisting of 3 x 3 right bottom blocks of all matrices of gI™'' and it fixes 
every element outside this subgroup. Hence, by Lemma |3l ip'^'^ is class preserving. Let 
A2 denote the subgroup of Aut(Gn'"'') generated by {ip'''^ | < fc, £ < to — 1}. Thus we 
again get p™' class preserving automorphisms of G^™'' . Notice that Ai and A2 intersect 
trivially. 

Now, fix J 6 {2, . . . , n — 2} and £ G {0, . . . , to — 1}. Then conjugations 

give inner automorphisms. 

Since the set of these inner automorphisms intersects trivially with Ai and A2, it 
follows that 

I Autc(G(,"))| > p2™'+'"(«-3)^ 
Hence | Autc(Gl™^)| = p2m^+mn-3m^ I inn(G^"')| = we have 

|Outc(Gl"'))| =p2™(m-l)^ 

Since the exponent of 72 (Gn'"'') is p, it follows that Autc(GTi'"^) is an elementary abelian 
group, and therefore Outc(G^™'') is also an elementary abelian group. This completes the 
proof of the theorem. □ 

4. Open problems on normal automorphisms 

In this section we mainly talk about the group of normal automorphisms of G, which 
consists of automorphisms of G fixing all normal subgroups of it. 

Normal automorphisms for various classes of groups were investigated intensively. We 
recall some of the results obtained in this direction. 

By [Ml [15] if G is free non-abelian group, then each of its normal automorphism is 
inner. Thus 

Inn(G) = Autc(G) = Aut„(G). 

Let P„, n ^ 3, be the pure braid group. It was proved by Neshchadim [191, that any 
normal automorphism of P„ is inner. It is well-known that P„ is a semidirect product of 
certain free groups. 

Problem 1. Let G — Fk \Ff , where k,£ ^ 2, be a semidirect product of free non-abelian 
groups. Is it true that every normal automorphism of G is inner? If it is not true in 
general, then is it true in the case when acts on Ff^ by identity modulo the derived 
subgroup ? 

Romankov [JT proved that if G is a free non-abelian solvable group, then Aut„(G) = 
Inn(G). Normal automorphisms of pro- finite groups were studied in [10] . Normal auto- 
morphisms of free two-step solvable pro-p-groups were investigated by Romanovskii and 
Boluts' in [T]. They described the group of normal automorphisms and proved that it 
is bigger than the group of inner automorphisms. Romanovskii [22] proved that every 
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normal automorphism of a free solvable pro-p-group of solvability step > 3 is inner. It 
was shown by Jarden and Ritter that all automorphisms of absolute Galois group 
G(Q) are normal and all normal automorphisms are inner. 

Problem 2. Describe the quotient Aut„(G')/ Inn(G') for G such that 
Aut„(G) ^ Inn(G), and the quotient Autc(G)/ Inn(G) for G such that Autc(G) i= Inn(G). 

The following facts are known [7 . If G is nilpotent, then Aut„(G) is nilpotent-by- 
abelian. If G is polycyclic, then Aut„(G) is polycyclic. In particular, if G is finite 
solvable, then so is Aut„(G). 

Problem 3 (Endimioni 5 ). Does the last statement hold true without finiteness hypoth- 
esis? 

Endimioni 'F proved that the group of all normal automorphisms of a metabelian group 
is solvable of length ^ 3. Moreover, he gave an example showing that this statement can 
not be improved. Indeed, Aut„(yl4) = Aut(y4.4) = S4, where A4 is metabelian and S4 is 
solvable of length 3. 

It was proven by Robinson [20 that for any finite group F, there exists a finite semi- 
simple group G such that Aut„(G)/ Inn(G) contains F as a subgroup. 

Problem 4. Does the last statement hold true without finiteness hypothesis? 

The following example shows that there exists an abelian group G such that Autn(G)/ 
Autc(G) ^ 1. 

Example 1. Let G = G = (a) be an infinite cyclic group. Obviously, Inn(G) = 
Autc(G) = 1, and automorphism ip : a 1 — > a^^ belongs to Aut„(G). Each subgroup 
of G is normal and ip sends each of its subgroup to itself. Therefore, Aut„(G)/ Autc(G) = 
Aut(G) ~ G2 is the cyclic group of order two. 

The following example shows that there exists a non-abelian group G such that Aut„(G) / 
Aut,(G) ^ 1. 

Example 2. Consider G = (a;, y | a:^ = y^). It was pointed out by Neshchadim |T9] that 
the automorphism 



is normal, but not inner. Let us show that this automorphism is not class preserving. 
Indeed, ip send central element to x^^. Obviously, and are not conjugate, 
because g^^x^g — x^ for any g £ G. Therefore, Aut„(G)/ Autc(G) 7^ 1. 

For p,q € define the group 



If p > is odd and < q < p with (p, q) = 1, then G{p, q) is a fundamental group of (p, q)- 
torus knot complement in the 3-sphere. Moreover, G(3, 2) ~ ,63, where _B„ is the braid 



group on n strands. Neshchadim jlSj proved that every normal automorphism of Bn is 
inner. As we mentioned in Example^ Aut„(G(2, 2)) / Inn(G(2, 2)) and Aut„(G(2, 2)) ^ 




G{p,q)^{x,y I xP^y'^). 



Autc(G(2,2)). 
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Problem 5. Describe p and q such that Antc{G{p,q)) =/= Inn(G(p, q)). Analogously, 
describe p and q such that Aut„(G(p, q)) =/= Inn(G(p, q)). 

The following problem is regarding knot groups. 

Problem 6. Do normal (or class preserving) automorphisms of a knot group are all 
inner? This is true for the trefoil knot group 0(3, 2). 

It is known that for free nilpotent group of class 2, every automorphism is tame (an 
automorphism is said to be tame if it is induced by an automorphism of a free group 
under the natural homomorphism) . It was proved by Neshchadim |17| that every tame 
normal automorphism of Fn/^i{Fn), where n ^ 2, is inner. Moreover, he gave a complete 
description, in terms of generators and relations, of the group of normal automorphisms 
of a free nilpotent group of class 4 with arbitrary number of generators. 

Problem 7. What can one say about tame normal automorphisms of 
Fn/jmiFn) if n ^ 2 and m ^ 5? 

Appendix A. Alternative proof of Theorem [T] 

The following proof of our Theorem [1] is suggested by N. S. Romanovskii. 
Alternative proof of Theorem [TJ First we show that all class preserving automor- 
phisms of r„.„ = \JTn{K) are inner. Since the group \JT2{K) is abelian, therefore 
Autc(UT2(if)) = Inn(UT2(i4r)) = 1. By inductive argument, suppose that Autc(UT„_i 
(K)) = Inn(UT„_i(if)) for some integer n > 3. Let G = UT„(JC). Define two subgroups 
A and B of G as follows. 

A = {{ttij) e G I = for all i < n — 1}, 

and 

B = e G I - 0}. 

Notice that A is an abelian normal subgroup of G generated by subgroups tn j{K), where 
1 < j < — 1, and B is generated by subgroups ti^i^i{K), where 1 < i < n — 2. The 
group G is a semidirect product of A and B. Thus B = G/A and every g ^ G can be 
presented as g = ba, where b G B , a ^ A. 

Let (fi G Autc(G). Consider the action of ip on the generators of B: 

V(^»,j-i) = ^M^r « = 2, . . . , n - 1 

where bi-i £ B, a,;_i G A, and ti^i-i = Then ip induces a class preserving 

automorphism ^ of i3, where 

Wt.i-i) = ^M-i' « = 2, . . . , n - 1. 

Since G/A = B ^ UT„_i(i^) and Autc(UT„_i(i4:)) = Inn(UT„_i(ii')), it follows that 
Tp S Inn(i?). Then there exists an element b G B such that Tp is the inner automorphism 
of B induced by b under conjugation. Denote by /{, the inner automorphism of G induced 
hy b <E B under conjugation. Notice that /j^^ = /f,-i . Consider the action of f^^^p on the 
generators of the group B: 
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where Ui = ai[ai, bi]. Let us represent 

= tn,l{ai,i)tn.2{.(yi,2) ■ ■ ■ tn,n-l{(Xi,n-l) , 

for some aij G K. Using the commutator identity, one can easy check that 

J-Oi _ J.*n,l(ai,l)tn,2(a;i,2)...tn,n-l(ai,n-l) _ j.t»i,i+l ("i.i + l ) _ j. j. / \ 

''t+l.i ~ ''i+l,i ~ ''i+l,i — l'n,i\ (J^i,i+1)- 

Consider the following element of A: 

a = tn,2{ai,2)tn,3{a2,3) ■ ■ ■tn,n-lioin-2,n-l)- 

Then for every i = l,2,...,n — 2 the following equality holds: 

Hence the automorphism = fa^ff^^y^ of G fixes all generators of B. 

Consider the action of ip on tn ^-i G A. Since ip is class preserving, it acts as follows: 

Since A is an abelian normal subgroup of G, it follows that i^"^!"""^ = ^nn-v Presenting 

^n,n—l ~ ^n,n— 1 [^n,n— 1^ ^n— l] 

and using the fact G A, we can write 

^n"n-l = tn^n-ltn,lif^l)tn,2ip2) ■ ■ • iri,n-2 (/3ri-2 ) • 

It follows that /32 ~ (3^ = . . . = /3„_2 = 0. Indeed, for any 2 < fc < n — 2, we have 

1 = 1p{[tn,n-l,tk^k-l]) = [iTi,n-l^n,l(/3l)^n,2(/32) • • • iri,n-2(Ai-2), ifc,fc-l] 

= [tn,n-ltn,l{l3l) ■ ■ ■ t„,fe-l (/3fe-l)i„,fe+l (/3fe+l) . . . tn,n-2{Pn-2),tk,k-lY"'''^^''^ 

[tn,k{Pk),tk,k-l] 
= tn,k-l{Pk)- 

Hence p2 = 0. Thus 

V'(*n,n-l) = tn,n-ltn,l{Pl) = tn,n-l^ 

where c = Therefore, fixes tn,n-i- Moreover, c commutes with all 

generators of B, since c G Z{B). Hence f~^ip = idc- Thus, we proved that is a 
product of inner automorphisms of G. So (p itself is an inner automorphism of G and 
therefore Autc(r„,„) = Inn(r„,„). 

Now we proceed to prove the general case, i.e., Autc(r„^fe) = Inn(r„^fe), 1 < < n. 
Since Autc(r„^„) = Inn(r„^„) for all n (as we proved above) and T^^k is abelian for 
1 < A; < 3, by inductive argument we can assume that Autc(r„i^fc) = Inn(r„i^fe) for all 
m < n and 1 < k < m. So assume that n > 4 and 3 < fc < n. Let A' denote the subgroup 
of r„_fe generated by the subgroups tnj {K) , where n — k+1 < j < n—1 and B' denote the 
subgroup generated by the subgroups where 1 < i < n — 2. Here we denoted 

by tij{a) the image of element tij{a) £ G under the natural homomorphism G ^n,k- 
Notice that A' is an abelian normal subgroup of Tn,k and B' = Tn,k/A' = Tn-i,k- 

Let If e Autc(r„jt). Then using arguments similar to the case r„^„, we can find an 
automorphism ip of Fn^k which fixes each element of B' . After that we can prove that tp 
is an inner automorphism of r„_fe. Hence (p is an inner automorphism. 
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